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FOREWORD 



Recent attempts to describe the status of education in the 
United States have been plagued by a lack of comparable infor- 
mation €Q3out education in other highly industrial countries* 
Consequently, officials of the U*S* Department of Education, 
Center for Education Statistics, initiated a conference with 
pairticipatirtg countries of the Organization for Economic Co- 
operation and Development (OECD) to discuss possible approaches 
to developing comparable statistics* The conference was held 
November 3-6, 1987 in Washington, D*C*, and was attended by 
participants from 22 countries who have responsibility for the 
development of policy regarding the collection and reporting of 
information about education* 

The aim of the conference was to agree on a small set of 
indicators that the participating countries could pursue jointly 
over the next 5 years. To reach this goal, all countries would 
need to reach accv "d on the indicators and some countries might 
need to consider iW data collection " activities in order to 
report comparable cross-national indicators* 

A ni2mber of speakers with experience in previous cross- 
national coinparisons were invited to share their experience with 
the OECD member country representatives through papers and 
speeches* These presentations covered ideas about education 
indicators, methodological problems of making cross-national 
comparisons of outcomes of education, and thoughts on the 
measurement of education outcomes, resources, and context of 
educational practices* 

The participants were enthusiastic about the possibility of 
jointly developing cross-national education indicators but were 
unable to arrive at a consensus for immediate implementation* 
The summary of the results of the meeting, included in this 
collection, describes the need for further discussions on 
specific subject areas in order to arrive at working definitions 
for a set of indicators* Although agreement on specific indi- 
cators was not reached, the conference opened new discussions 
about steps that could be taken by OECD members to increase the 
availability of international comparisons on education outcomes, 
resources, and context* 

Participants of the meeting were asked to summarize the 
statistical activities underway in their countries on the use of 
performance indicators and to describe sources of statistical 
data on education outcomes , resources , and student context of 
education* Copies of the papers prepared by the speakers and 
country representatives can be obtained by writing to the 
National Center for Education Statistics (NCES) or to the OECD in 
Paris* The addresses of NCES, the OECD Secrr :ariat, and all 
participants are in appendix C* 
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REMARKS TO THE INTERNATIONAL CONFERENCE ON EDUCATION INDICATORS 



Emersoa J. Elliott 
Director ' 

Center for Education Statistics 



Since the United States proposed this conference^ it may be 
useful for me to comment at the outset on 1) how we view 
education indicators in the United States and their possible 
utility in an international setting and 2) what we would like to 
see emerge as a result of this conference. 

l^e United States is in the midst of efforts, discussed at 
previous meetings of OECD, to strengthen the rigor of its 
education system and increase the achievement of its pupils. 
Those efforts have raised questions for policymakers about what 
happens next. For example? 

• Will tougher cxirricula conflict with goals to reduce 
dropouts? 

• Wliat effect will new standards for teachers have on the 
demographic composition of the teacher work force? 

• Will pay incentives keep teachers from leaving the 
profession? 

Policymakers in our State capitals are realizing that such 
questions can be amswered only if data are provided to monitor 
the progress of education reform. Our State governors, in their 
national association, have initiated a project to follow such 
data for at least 5 years. Our chief State school administrators 
have called for compareible data across States. 

For several years, we at the Center for Education Statistics 
have pursued a data reporting project that describes the health 
of education by collecting information about 

• what our students know and what level of skills they 
possess; 

• the quality of educational processes, especially those 
processes that have strong correlations with outcomes; 
and 

• the educational needs of students and the policy 
context for education as expressed in government 
action. 



We 1;ry to distinguish "indicators", of the health of 
education from more traditional measures such as the number of 
students, size of school, or level of expenditures. We see 
indicators as statistical measures that show relationships such 
as 

• trends over time; 

• measures of one State compared with anotJtier or schools 
sharing similar characteristics compared with each other; 
and 

• comparisons of achievements against goals. 

Education indicators draw, where possible, on findings from 
education research. They serve as signposts or benchmarks. They 
can help provide a fuller understanding of how our educational 
systems are functioning by identifying the high spots or the low 
spots, both of which are the places we should examine closely. 



In the United States, we find keen interest in information . 
that places American measures side-by-side with those of other 
nations — not because we want to hold someone accountable, or want 
to. run an international academic oiympics, but because we expect 
to find differences that will give us something to ponder, that 
will enable us to question our own actions, that will let us 
assess whether a practice of another country might fit here, or 
that will cause us to reject them because they do not seem to fit 
our situation. 

Let me cite an example: 

The United States and Japan have recently completed exten- 
sive studies of each other's education systems. one major 
conclusion the U.S. Secretary of Education drew from this work 
was that we have been badly served by the American tendency to 
reject the lessons of Japanese education on the ground "the 
culture is so different." The Secretary found a dozen practical 
lessons from the study and " urged Americans to look for 
"principles, emphases, and relationships" that are compatible 
with American values. Among other lessons was one on the amount 
of time devoted to education — it is much greater in O'apan than in 
the United States. But that information was harder to find than 
simply looking at the "number of days in school" — it required a 
deeper look at days and half-days, out-of-school time and 
tutoring in "juku" programs, and the efficient management of 
class time in school. The information we gained from this study 
is very timely as Americar. schools look for ways to increase 
effective learning time for students. 

It is obvious from the papers prepared for this conference 
that many of us have questions about the utility of cross- 
national data, how that data might be used, the various ways of 
explaining reasons ,for differences, and other matters. I said 



"many of us" because the United States as a country also has such . 
questions and, in fact, several of these questions were included 
in the briefing paper sent to you for this meeting. 

But I have seen the care that the International Association 
for the Evaluation of Educational Achievement (lEA) has given to 
suck questions internationally, developing, for example, a 
concept called ••opportunity to learn" that permits comparisons of 
achievement by taking differences among curricula into account. 
.Moreover, I can assure you that most of these questions, are ones 
regularly raised when we gather data from our States and report 
them — data such as populations, goals, values, and resources, and 
how they are accounted for and reported. 

I have learned that when government leaders press their 
questions repeatedly enough, there can be a response. Such 
questions can be addressed. 

Let^s set out to do that. 

A year ago — even 6 months ago — we thought we would be 
fortunate if 6 or 8 nations, perhaps 12 or 15 participants, - came 
to this meeting. We have three times that many, and I am both 
amazed and pleased. {1 hope OECD is also.) Because of the 
strong interest demonstrated by your presence here and the papers 
you have prepared, I think it i^ realistic to state a goal for 
this conference. 

The goal is that we agree here to embark on an exchange of 
data on measures of the progress and trends (or "health") of 
education. More specifically, such an agreement might: 

• include those nations that express interest, 
although perhaps not all those represented here 
will wish to join in; 

• cover several indicators, perhaps 5 to 7, with 
additional ones selected for further examination; 

r commit participating nations to support any necessary 
technical staff work to deal with details that will 
inform users about the degree of comparability in any 
indicators reported across nations; and 

• call for a report to the OECD education committee 
meeting in December as to the results of our work this 
week, with recommendations for appropriate, continuing 
involvement of OECD in this activity. 

This ma:^ add to the annual data collection of OECD, but I 
want to emphasize that there is intended to be nothing in these 
suggestions that would require additional data collections in any 
country unless it wished to do so. The point is to use 
information that is already available or is scheduled to become 
available. 
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I want to make this suggestion today ^ at the beginning of 
our discussions^ so that each of us can think toward the next 
steps that might be taken* I 2un sure the views of all of us will 
be shaped by these sessions, and, as the discussions proceed, we 
will achieve better understanding. 



U.S. EDUCATION REFORM AND INTERNATIONAL DATA 



Che«,ter E, Finn, Jr. 

Assistant Secretary for Educational Research and Improvement and 
Counselor to the Secretary 



I. am pleased to be talking with a group that could lay the 
groundwork for increasing the quantity and quality of information 
about the schooling of youngsters around the world* Americans 
are extraordinarily interested in this* International studies of 
education have served as a stimulus (sometimes on the level of an 
electric shock) for some of the education reforms currently 
underway in the U.S. We believe they can catalyze and • inform 
more such. 

• 

The 1980s have been a time of extraordinary educational 
ferment and change in this country. We have been swept up in a 
massive education reform movement for over half a decade, and the 
momentum shoys. no sign of 2ibating;- in fact, it . seems to be 
accererating. Americans consistently put education among the top 
three or four most important issues facing the Nation, and 
Southern legislators recently pegged education as ' the number one' 
domestic policy concern among voters. In September, .we witnessed 
an unprecedented event: the first-ever debate among Presidential 
candidates to focus exclusively on education. 

Education has never been more important to Americans, and 
the need for systematic feedback- on the performance of our 
education system has never been greater. Before elaborating on 
this need, and some of the ways that cross-national data can help 
fulfill it, I'd like to recapitulate a few of the changes that 
have occurred in American education over the past decade. It is 
functional to categorize six facets of the reform movement 
currently underway. I also tend to think of them as six fronts 
in the war on educational mediocrity. 

The first front got started in the 1970s, when schools began 
marching their students back to the basics. This renewed 
emphasis on the three Rs (reading, writing, and arithmetic) 
carried over into the 1980 's and is still very much with us. 
States and districts have fallen into line by raising standards 
and testint,^ minimum literacy and math competencies, first of 
students, then often of teachers as well. We still have a 
distance to go here, but we are making respectable progress, so 
much so that attention is shifting increasingly to reasoning, 
analyzing, and the other higher order skills that rest atop the 
basics . 
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On the second front, we encounter the many issues pertaining 
to school professionals (teachers, principals, and others) : 
where to find them, how to prepare and certify theia, how to tell 
if they are good at what they do, how and at what levels to 
compensate them, how to make them more professional, how (if at 
all) to distinguish among them, what to do about bad ones, how 
teachers should relate to principals, and so forth. The most 
visible effort in recent years has been the Carnegie proposal to 
create a national — but not administered by the government — 
teacher certification board. Another important development is 
New Jersey's statewide "alternate certification" program for new 
teachers, which is attracting hundreds of well qualified persons 
into the classroom — individuals who had not prepared to be 
teachers when they attended college. A growing number of 
education reforms are calling for a fundamental restructuring of 
the schools in America, a change tJiat would focus even more 
attention on the professionalism of the people in them. 

The third front is school effectiveness. Research has 
revealed what many of us suspected all along: that certain key 
elements are usually found in successful schools— char'^cteristics 
such as a shared sense of purpose, an ethos of achievement, 
vigorous team spirit, and a clear, coherent curriculum. And 
effective schools are well led, usually by a principal who has a 
clear vision of the school's mission; who successfully arti- 
culates that vision to teachers, community and students; and who, 
through his or her daily routines, creates conditions to make 
that vision a reality. 

The fourth front is educational equiV.y, which has been 
conspicuous on the policy horizon for over 20 years but which has 
lately raised its profile. Special education i« not new; dropout 
prevention programs are in place in many school systems; so are 
compensatory programs of many kinds. In the past, however, the 
dominant equity concern was furnishing greater " access " to 
educational resources and services for students who may not 
previously have had their share. But we have come to realize 
that access does not ensure achievement, and that resources do 
not guarantee student learning. Outcomes, after all, are what 
matter. Providing truly equal opportunity for all youngsters 
means setting serious standards for ail students and doing our 
utmost to enaUale them to achieve those standards. 

The fifth front is the newest; it has to do with content: 
facts and concepts. . . knowledge. This concern is deepening as 
the evidence of ignorance mounts. In 1986, the first assessment 
of U.S. students' knowledge of American history arid literature 
yielded startling results: 

• One in three of our high school juniors thinks 

Christopher Columbus reached America after 1750. The 
same number have no clue when the Declaration of Indepen- 
dence was signed, or who Aesop, Atlas, or Cain and Abel 
are. 



• Two cut of five cannot guess the meaning of "checks and 
balances, a central concept to our system of government. 
Approximately the S2ime portion, 40 percent^ don^t know who 
wrote The Iliad ^ or what The Scarlet Letter Moby 

I2isK, or 2h£ £sd Ba<aq? CgVir^q^ are about. 

• Barely one out of three can identify tha 50-year period 
surrounding the time in which our Civil War was fought. 

• These kinds of questions produced average scores of 55 
percent in history and 52 in literature — obviously 
failing grades.* 

There is r. growing consensus in this country that the facts 
and concepts of history, geography, and literature ought to be 
part of what our children learn in school. So should essential 
knowledge of other core subjects. Many States and school 
districts in the United States have raised course requirements in 
the sciences and mathematics, among others, which is a step in 
the right direction. But it is not enough to earn four credits 
of English, three in mathematics and three in science. We need 
to make sure that all students master specific cognitive skills 
and ]uiowledge at each grade level. This brings me to the sixth 
front of education reform in the U.S.: accountability. 

In September, Secretary Bernett called on the nation's 
leaders to make accounteODility the icheme of t a year's education 
debate « Americans already want greater accountability in the 
education system. Over 70 percent say they want student achieve- 
ment test results reported so that comparisons can be made State- 
by-State and school-by-school. Three out of four believe that 
student performance on academic achievement tests should be a 
condition for promotion from one grade to the next. 



Accountability involves two elements: 1) information as to 
whether one^s education goals are in fact being obtained and 2) 
consequences linked to the results • It means consequences not 
only for students, but also for schools and for the people 
working in them and the policy units of which they are a part. 
The idea is that schools should be rewarded for increasing the 
academic gains of their students. And schools that fail to 
produce measurabla *iains in students' learning need to be 
changed. 

A j/rowing number of States hold public ceremonies in which 
governors and other State officials present awards, trophies, and 
sometimes even cash bonuses to schools that have produced 
outstanding gains in student achievement:. Even more States have 
laws that permit them to withhold funds or remove officials from 
scho;:^'' or dS5^r.tricts where student performance remains poor. 

ne United States, the main responsibility for such 
ini' ions and incentives lies with the State and local 

go\ lii our country, education has always been a State 
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and local responsibility. Each of the 50 States sets its own 
education policies, and some 15,000 school districts carry out 
those policies (hot to mention nearly 28,000 private schools). 
The national government's main role, at least wit>h regard to 
elementary and secondary education, is to provide periodic 
checlcups on the health of the overall system and to report these 
findings to educators, policymakers, parents, and citizens. 



* The U.S. Department of Education collects statistics and 
other forms of information on various features of the education 
system. By far the most important of these have to do with 
student outcomes, student achievement, student learning. This is 
the indispensable baseline for accountability, and there is a 
major push in this country to upgrade the National Assessment of 
Educational Progress, one of our main sources of information on 
how well our students and schools are doing. We have proposed 
changes in the national assessment that would allow comparisons 
of student performance to be made among States. This is a 
radical, historic change. But we would like to take it a step 
further. We want to make it possible for States to compare their 
student performance with that of other countries. If California 
wants to appraise its education results alongside Japan's, it 
should be able to. Of course, we want to compare student 
achievement nation-to-nation as well. Several other countries 
.are using our National Assessment of Educational Progress science 
and math questions to assess their students' performance. This 
indicates to me that State-to-nation and nation-to-nation 
comparisons are riot unthinkable. Furthermore, we have the lEA 
precedents. 

Such comparisons supplement data from domestic studies and 
help us to monitor trends and changes over time. Domestic 
information alone simply does not provide a big enough picture. 
International information establishes the larger context; it puts 
domestic data in perspective. A nation may take satisfaction in 
the math gains its students are making, for instance. But there 
is little cause for pride if students in other couhtries are 
improving at twice the rate. 

Americans have not always appreciated international informa- 
tion on education; at times we have been self-absorbed, even 
insular. We value it now, however, and we hunger for more 
information than is currently available. Education achievements 
in other countries provide benchmarks ^ standards, and goals for 
us. Japan, for instance, does something many Americans had not 
thought possible. The U.S. Department of Education's 2-year 
study found that the Japanese graduate an extraordinarily high 
percentage of their students from secondary school while simul- 
taneously producing a very high average level of academic 
achievement. This study challenged our conventional wisdom. 
Contrary to the assumption of many in the United States, equality 
and excellence both can fc^ achieved within the same system. 

Another example where an international study "removed our . 
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blinders" is the lEA's Second International Mathematics Study* 
It bohf irmed something we knew, that achievement follows content 
coverage and that students tend to learn what is taught. But the 
mathematics s€udy also revealed something that many Americans 
never considered possii>l3 — that students can be taught complex 
mathea^atics at a relatively early age* 

Cross-national research sometimes widens our horizon and 
suggests nonincremehtal means of improvement* The discovery that 
a humb€ir of other cotmtries devote more time to schooling has 
helped stimulate debate in the United States about lengthening 
tiie school day amd year* 

■ International comparative studies can also indicate areas 
that need; attention. The 1972 lEA reading and literature 
studies,* for instance, showed that a major weakness among U.S. 
Secondary students is con^rehension. Prelimihary findings from 
the new lEA science study show that U.S. students are far below 
their English amd Japemese counterparts^ With apologies to my 
British^ friends, let me observe that the U.S. is accustomed to 
lagging behind Japan, but we were, frankly, surprised by our 
scores- in relation to the United Kingdom. 

International comparisons of student performance generally 
stir debate in this country. They sometimje^ trigger initiatives 
and activities at the State arid local levels. In the aftermath 
of the-*' Second International Mathematics Study, for instamce, a 
consprtitim of school districts in the State of Maryland was 
formed to examine the study results and to initiate reforms. An 
orgemization in the State of Minnesota, the "Mathematics Quorum," 
infoxrmed each school district in the State about the conclusions 
of the international study. Three other States — Virginia, 
Florida, and Iowa— ^conducted their own surveys of mathematics to 
compare themselves with other countries. The results helped 
Floridians develop a new geometry -curriculum > 

International information has helped to fuel efforts and 
debate about education improvements in the United States. I 
cannot help but think that it does (or could do) the same in 
other industrialized democracies, most of which are OECD members. 
Moving forvtard in matters of social welfare, economic growth, and 
cultxural vitality is something that concerns all of us. And 
progress in each of these domains depends in no small part on 
improvements in education. At a time when technology has shrunk 
the giobe-- when TV, telephones, and jet travel have turned us 
into yirtual next-door neigitt)ors-->we need, more than ever, to 
learn from each other, to benefit from each other's experiments 
and experiences, triumphs, and errors. We need to undertake 
collaborative efforts whenever possible. 

Secretary Bennett and I are committed to this. The U.S. 
Department of Education plans to publish a study of teacher 
salaries in 15 countries and a volume of scholarly background 
papers, and other original research from the Japan study. Early 
in 1988, we hope to j^iiblish a study of how different countries 



14 



otfer parents various kinds of- choices among schools, choice is 
of particular interest to us; it is a natural mechanism for 
bringing accountability into the system, for making schools 
responsive to their markets. But this is just the start. 

The upcoming reading literacy study by the lEA could set a 
new precedent for international cooperation if international data 
and domestic data axe deftly coordinated. We hope that the 
samples and content from this study can be linked to our own 
national assessment. 

We need more linkage and coordination like this. And we 
need more data on education around the world — more data and more 
kinds ;6f it. We want a reliable way to chart education's 
percentage of the gross national product in each couhtry — not 
just government expenditures on education; but all expenditures 
on education. Why can't this be done? It is among the most 
fundamental points of reference for comparing education systems 
and their relative performance and productivity. Another 
international blind spot is the curriculum. Can we not do a 
decent coa^arison of course content, subject-by-subject, across 
nations? 

Some of the reasons why we lack international data are 
understzmdzUsle. Countries organize their systems differently and 
measure things differently. Data, on school dropouts, for 
example, are- difficult enough to collect within the United 
States; it's exceptionally hard to get the 50 States to agree on 
definitions, on the elements that should be measured. Difficult 
as this may be, it is vital that we work out a way to do so", and 
not just here in the United states. Attrition and completion 
rates are among the essential indicators of health in any 
education system. 

But if some of the reasons we lack international comparisons 
are reasoneUale, at least for the time being; others are disrepu- 
table. Too many countries — the United States has been vulnerable 
to this at times — have been uncooperative, resistemt, uninter- 
ested, or simply lazy. We need joint efforts to overcome these 
impulses and galvzmize support for more valid, detailed, varied, 
and useful international information on education. 

In the field of education, the organization best positioned 
to accomplish this (at least among the industrialized demo- 
cracies) is the OECD. It already generates some useful data. We 
are glad to have these, but they are not enough. Should not a 
lax^er share of OECD*s energies amd resources be allocated to 
efforts to increase the quantity and quality of information on 
education around the world? lii a word, yesi This responsibility 
ought to belong to OECD; no other entity is so well situated or 
qualified. 

It is a difficult task, to be sure, both for the inter- 
national organization that takes on such responsibilities and for 
the member countries that get involved. The United states has. 
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^ * at times r been culpable of tardiness and . incompleteness when 
filling put even fairly simple questionnaires sent by bECD* We 
vow to dp better* We hope our colleagues will take the same 
oath. 

But improving the quantity and quality of international 
information on education is not just a matter of completing forms 
emd gathering nuinbers; it is also a matter of reaching agreement 
on which indices and gauges and definitions should be used — and 
how tiiey can be used in ways that respect the differences* in our 
several nations* education systems emd that are useful also for 
intematipnal purposes. 

The. single most importemt yardstick for the performance of 
any education system is student outcomes, student achievement, 
and student learning. I think we all agree on that. Certainly 
it is information about outcomes*«*^including comparisons aunong 
outcomes-*-»that elected officials emd policymakers primarily 
demand. But we also heed statistics euid research and measures of 
many more dimensions of our education systems-*«*such as curriculum 
content, school orgemization, attendance, learning time, parent 
involvement, homework policies, retention and completion rates, 
enrollments in advanced courses and in college, the uses to which 
education revenues are put, performcmce in college, and other 
features of education systems. We need to gauge and describe the 
gaps between the intended cvurricula* the implemented curricula, 
ahd tiie attained cnirricula. We also need to look for links 
between student outcomes and those varied^les that parents and 
educators C2m control. 

I refer to Harold Stevenson's finding that American parents 
are apt to attribute their child's learning to innate ability, 
whereas Asiam parents more often believe it to be mainly a result 
of hard work (this, of course, is part of the reason Asian 
students learn more). Studies such as Stevenson's are doubly 
importamt, because they not only indicate differences in student 
achievement, but also probe factors that may explain pairt of the 
reasons for those differences. The Stevenson study points to a 
variable that lies at least partly within human control: 
attitudes are something parents and others can do something 
about, especially if they learn from international comparisons 
that different attitudes can be associated with better results. 

I would like to see more studies like Stevenson's, and more 
analyses of data, analyses across subgroups, institutions, 
regions, nations, and other geographic and cultural boundaries. 
Such analyses can reveal weeUcnesses in an education system. They 
C2m also highlight features that could be most responsive to 
improvements, that is, those most likely, if modified, to yield 
increases in student outcomes. 

This, finally, is the reason we have gathered: to determine 
what information is vital to all of us in our efforts to improve 
the education system and boost rtudent learning in our coiintries. 
Not everything that is important to find out is amenable to 



measurement ; there are subtle qualities of mind and spirit that 
jpsyphometrics cannot capture or express. However/ much that we 
want to know can be quantified and gauged, and it should be« The 
whole field of testing and measurement is making great strides 
and will maUce more. He must not forget the immeasurables , but we 
must not slight what can be measured. 

I urge you to consider all the possibilities. Each interna- 
tional, indicator adds a piece to the picture of education. The 
more pieces we have, the clearer the pictiire becomes. A clear, 
accurate picture is critical to developing solid, effective 
education policies. 

The indicators we can agree on and then generate will help 
determine the view of education we see and base our decisions on. 
These few days in Washington offer an opportunity to lay the 
groundwork that coiild lead to improvements in education for 
millions of children around the world for years to come. I wish 
you much wisdom and a measure of boldness! 
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SUMMARY REPORT ON CROSS-NATIONAL EDUCATION INDICATORS 
OECD Secretariat 



I. INTRODUCTION : BACKGROUND AND CURRENT INTEREST IN 
INDICATORS 



1. Under the initiative and invitation of the Uj.S. Education 
authorities/ and jointly organized with the OECD Secretariat/ the 
international conference on education indicators took place on 
November 3-6, 1987 in Washington D.C. It was attended by 
national representatives of 22 member countries, as well as 
invited experts emd observers from both the United States and 
other OECD countries. The reasons for the priority given to this 
areia by the U.S. authorities were underlined by the U.S. 
Department of, Education officials, Peter Greer, Deputy Under- 
Secretary for Iiltergovemmental and Interagency Affairs and U.S. 
member of the Education Committee, and Emerson Elliott, Director 
of the Center for Educs^tion Statistics, in their opening 
addresses. The need for information and bench-marks that might 
allow comparisons across countries on how well education is 
functioning in each was seen as an integral component of the 
improvement of its quality for all countries included. Though it 
emerged in the conference that there are national differences of 
priority concerning how schooling is to be improved, the 
importance of that broad goal and the need for information, to 
support it were echoed repeatedly by country representatives. In 
his introductory remarks, Elliott also indicated the de^jire of 
the U.S. authorities for the meeting to reach concrete 
conclusions in the form of agreement on the need for further work 
in this field and on the areas on which this work should focus. 

2. These themes were taken up and elaborated by Chester E. 

Finn Jr., the U.S. Assistant Secretary for Educational Research 
and Improvement, Counselor to the Secretary, and U.So member of 
the CERI Governing Board, in his address on the evening of the 
first full day of the conference. He addressed the issues facing 
education reform in the U.S. organized around six "fronts" of the 
reform movement ~ curriculiim standards, school professionals, 
school effectiveness, equity (of outcomes and not only access) , 
content knowledge, and accountability. The latter implies both 
information and consequences. There is need, he said, for more 
and better data on the progress of education in a wide array of 
areas. Data are needed at the individual student, classroom, 
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school, district, state, and national levels, in order to provide 
feedback to planning improvement. ke especially emphasized the 
value of using international information in order to establish a 
context through which to understand the domestic situation and to 
allow countries to learn from each other. The OECD, he regarded, 
is best positioned to undertake this, and should make it a 
priority of their educational activities. 

3. George Papadopoulos , of the OECD Secretariat, placed the 
conference in the -context of the long-standing commitment of 
Member countries to develop cross-national educational statis- 
tics. But it signaled a re-emergence of attention to indicators 
as such after the previous period, more them a decade before, of 
keen interest in this field. That period had been notable in OECD 
work by the agreement of high level officials at the Paris 
Conference on Educational Growth in 1971 to make goals more 
explicit and progress towards them measurable, following a decade 
of very rapid expansion, and the publication of Indicators of 
Performance of Educational Systems in 1973. Several delegates 
also placed indicators in the context of previous international 
work on this topic, welcoming renewed attention to education 
indicators and noting that the previous interest witnessed in the 
late 60s and early 70s had not been sustained in the following 
years. The major question for the conference and any subsequent 
work, Papadopoulos concluded, was whether the undoubted national 
interest in developing indicators could be translated into 
measures that would be meaningful internationally. 



II. INDICATORS : CONCEPTS AND MEASUREMENT ISSUES 

What are education indicators? How can they be used? 
What kinds of issues do they raise for various groups 
involved in education? What are the optimal components 
of a system of indicators? 

4. This session was divided into three parts, with 
introductory papers by: 

- Jeannie Oakes, The Rand Corporation, California, 
Education Indicators — Concepts, Types, Uses; 

- T.N. Postlethwaite, University of Hamburg and former 
Chairman of lEA, Methodological Issues on Indicator 
Development on the International Level 

- P.H. Halsey, Department oi' Education and Science, 
England, Education Indicators: Concepts, Measures, and 
Educational Policy in England and Wales. 

The respective discussants for these presentations were 
respectively Denis Meuret (France) , ftory O'Connor (New Zealand) , 
and Kjell Eide (Norway) . 

5. Jeannie Oakes, in her presentation, outlined some of the 
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different definitions and approaches that can be adopted for the. 
development of indicators and some of the issues and problems 
that laust be confronted in that development. Whether as single 
or as composite statistics^ indicators cover key aspects of any 
education system. They may be developed ac benchmarks of 
progress/ as ••bell weathers" to help show likely future change ^ 
as: descriptions of central aspects of schooling^ or as policy- 
relevant or problem- focused inforaatipn. Though they aim to 
allow comparisons . (across time^ parts of the system ^ or 
coxiritries) , they cannot be unequivocal nor are they devoid of 
sxabjective judgment. This subjective element derives in part 
from the choice of the overall model or framework that determines , 
how the different parts are seen to relate together to form the 
whole/ and in part through value judgments about desirable 
outcomes and the best means cf realizing these. She emphasized 
that, while there are formidable technical and political problems 
to be overcome in this field , indicators can bring new and 
relevant knowledge to bear on outstanding educational and 
political issues. 

6. The discussion that followed covered a wide range of the 
complexities inherent in the development of educational 
indicators. Experience from a number of countries showed that 
current interest in the information furnished by indicators 
derives not only from the perceived inadequacy of traditional 
statistical ' measures but from increased external pressure for 
greater accountability. The need for measures that respond to 
thi& pressure should, some warned, be carefully defined according 
to the level of the education system or decision-making process 
involved. In this regard, the distinctions were made between 
global and disaggregated indicators and between measures that 
indicate how discrete aspects of the education systems are 
functioning and evidence of how these partial elements contribute 
to the overall well-being of that system. The close relation 
between considerations of quality and equality were identified as 
important in at least two respects: first, that indicators of 
outcomes should properly take account of the differential social 
intakes of pupils from one school or region to another, and of 
the differing environments outside schools that influence those 
outcomes; second, that measures expressed as averages should be 
complemented by those of the distribution of outcomes across the 
different social and ethnic groups that compose the school 
population. 

7. Certain fundamental questions were raised in this session 
that arose repeatedly throughout the conference. How, if 
indicators are only one among several a^pproaches to the 
assessment of quality and priorities for its improvement, can 
decision-makers avoid over-reliance on any single indicator, 
particularly as the political process is a far more complex one 
than the mechanical weighing of statistics and results? Should 
the range and number of indicators that might be developed 
ihtematipnally be broad or else restricted to a limited number 
in a few key areas? Several delegates argued in favor of a broad 
approach, both in order to accommodate the breadth of existing 



educational objectives within countries and to allow choice 
between different measures according to divergent national goals 
and priorities. On the other hand, the view was also expressed 
that considerations of feasibility and of the uneven availability 
of information across Member countries suggest that efforts 
should focus on a few key indicators. 

8. A dominant theme of Neville Postlethwaite's presentation 
pn the methodological aspects of indicator development was that, 
Important though the technical problems and caveats are/ the 
paramount considerations in the international field are the 
conceptual framework to be adopted and the identification of the 
uses to whicli any measures ar^ to be put. He directed many of 
his reimarks to those indicators and measures that cannot be 
derived from regular, administratively-gathered statistics and 
which require surveys based on (sufficiently large) samples. 
Thorny methodological issues of comparison must be addressed and 
resolved. These include the need to ensure that "like is 
conrpared with like" (thus raising the question, for example, of 
whether modal grade level or age of student (or both) are to be 
used in defining the sample population) and that the requisite 
variables, even those that seean at first unproblematlc (e.g. 
"participation" or "truancy"), be defined precisely enough to 
allow comparisons to be drawn. Students' achievement, despite 
the problems involved, were seen by Postlethwaite to be 
significantly .more amenable to international inquiry than their 
attitudes:. He elaborated some of the techniques that can be 
adopted in constructing tests that sufficiently reflect the 
curriculum' of each country, while allowing relative cross- 
national mastery of knowledge to be assessed. Measures can also 
take account of differential retention rates in the form of 
"yield measures" of achievement. 

9. The ensuing discussion covered both the methodological and 
political dimensions of indicator development. The particularity 
of practices and arrangements in each country was. seen to imply 
that cross-national comparisons even of concepts such as enroll- 
ments or advancement rates could not be lightly undertaken. One 
delegate suggested that an implication of this is that further 
work should concentrate primarily on methodological issues rather 
than report actual results, The question of how broad should be 
the range of potential international education indicators again 
arose. Country differences were rejported as to the existence of 
a national curriculum that might facilitate the delineation of 
clear bench-marks of knowledge,, at least within a single country, 
and in several there is a tradition of decentralised curriculum 
planning > even if that is in some places open to review or 
change. The e&bsence of a national curriculum, it was suggested. 
Implies that particular care should be taken in defining inter- 
national indicators of student outcomes. The view was also 
expressed that goals and cultures are so specific to each country 
as to render invalid cross-national measures of achievement. 

10. Despite these problems, in turning to the political uses 
and dimensions of education indicators, it was agreed that 




policy-makers would in any event usa information and make 
r/omparison^ using available indicators, irrespective of the 
riders provided by the research community* Th€^refore, it was 
suggested thi^se indicators should be as extensive, reliable, and 
valid ilB possible^ Part of on-going educational research 
activity might be set aside and closely tailored to the needs of 
policy making; The Jnhirently political nature of this subject 
was ^Iso described iri terms of "system , dynamics" — that 
education, as a human, creative enterprise^ will always respond 
to ^±te- information that is generated about it, whether positively 
or negatively. That there should be response was not seriously 
questioned the^ object of as^eissing how well schools function 
being, after all, to identify and implement possible 
improvements* Yet, there can nlso be distortions and negative 
responses and those should, as far as possible, be avoided. One 
delegate did suggest, however, tJint ^ s^irice there is no obvious 
decision-making authority corresponding, to the field of cross- 
national inquiry^; the risk of distortion is thereby reduced. 
Examples of possible distortions were given at various points 
during the conference. Drawing the distinction between 
information that is politically sensitive/relevant and that which 
is primarily of technical interest, several pointed to the 
(perhaps not unexpected)^ fact that there is considerably more 
demand for the former than the latter. In other words, there is 
most interest in precisely those fields where interpretation is 
likely. to be contentious. 

11. '. Philip Halsey's presentation concentrated on the national 
policy context of England and Wales At the national level, a 
high current priority in that education system is the 
establishment of national standards and objectives. This is 
based on the dual premise that present achievement levels, while 
perhaps generally satisfactory, are neither as high as they could 
he nor as they need to be given the demands of the modern society 
and economy. Many of Halsey's remarks, however, addressed 
information and indicators needed at the school level, since a 
key assumption of present policies is that school improvement is 
crucially dependent on better local decision-making and enhanced 
parental choice, as well as more systematic assessment. This 
made it especially important, in his view, that that information 
be comprehensive. Correspondingly, his suggested model for 
indicator development, while sharing features with those proposed 
earlier by Oakes and others, differed significantly from them by 
locating the school at its centre, with information also needed 
on the input3, the legislative and organizational framework, and 
the outputs — each broadly defined. Like a number of other 
speakers, in this and other sessions, his conclusion was that 
recognized difficulties should not be a decisive obstacle to 
progress in this field. 

12. Several delegates returned to the distinction between 
global and disaggregated, school-level indicators to emphasize 
that, while closely inter-related, they are not identical. A 
further distinction drawn was between short- and long-term time 
frames and the related dichotomy of statistics and research. It 



17 

22 



W3is hbted that an international organization such as the OECD 
should be strategically placed to span the divide between 
responding to immediate demands for knowledge and the long-term 
development of that knowledge base. Speakers emphasized that not 
all policy-relevant questions are euaenaible to routine measurement 
and response. One objective for further work in this field 
might, it was said, be the production^ of guidelines for the 
research community so that "new versions of the wheel do not have 
to be repeatedly reinvented." The passage of time was identified 
as raising seme of the particularly pointed questions to be 
addressed in this area and not only because tlie research process 
in frequently too slow to generate immediately usable results. 
One speaker argued that, insofar as measures could be validly 
defined in order to establish trends in pupil performance, this 
in itself should suggest that education systems are not suf- 
ficiently responsive to the rapidly changing environmentr in 
which they are located. The more valid the measure, in other 
words, the less satisfactory the. responsiveness of schools. 



III. INDICATORS OF THE OUTCOMES OF EDUCATION 

What are the best sources of data on student 
performance in schools? What are the major issues 
regarding comparability of scores across countries? 
What other outcomes of schooling need to be explored? 

13. - ' This session was divided into two parts with introductory 
papers by: 

- Alan Purves, chairman of lEA, Student Performance - lEA 
Studies; and 

- Colin Power, Flinders University, Australia - outcomes 
- Other Possible Measures 

The discussants of these papers were R. charters d'Azevedo 
(Portugal) and Esse Lovgren (Sweden) , respectively. 

14. ■ Alan Purves presented a paper on the experience and 
expertise embodied in the lEA. It has significantly developed 
the assessment of student performance against a background of 
home, school, teacher and student variables as well as measures 
that can be used locally, nationally, and internationally. It 
has an ambitious long-term program with studies in different 
knowledge and subject areas already envisaged almost up to the 
year 2000 that will involve the development of new tests, 
indices,, and scales. He responded specifically to possible 
criticisms of the lEA enterprise, especially those that maintain 
it is insufficiently sensitive to the cultural and curricular 
particularities of different countries. He described the efforts 
undertaken to develop valid cross-national measures, and 
emphasized that standards are not imposed on countries through 
ISA's work. Rather, each country decides those levels or 
standards for -themselves. 
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15. As well ais discussing methodological and political issues 
^ of the kind already mentioned> the conference also addressed here 

a mmber of othsir wide-^ranging questions. Did the lEA process or 
mechanism offer a model for more extensive international activity 
in this field? Could lEA measures be readily converted into 
international indicators and could this be done without a major 
injection, of permanent staff and resources? Are cross-national 
measures of students achievement feasible and useful? Divergent 
nationikl views . were expressed on this point. Should the 
knowledge and skills measured relate .only to schools and school 
systems? ' Certain delegates regarded that definition of the 
field, particularly in a system such as the German dual model 
where vocational training and apprenticeship are an integral 
component of the education system, as overly restricted • A point 
that was underlined several times during the conference was that 
there already exists through ISA work a significant body of 
measures and expertise that would obviate the need to develop all 
indicators completely anew. 

16. Colin Power's presentation on indicators of outcomes other 
than those of stixdent achievement reiterated the general view — 
that there is a definite need for indicators and for more 
comprehensive information yet particular care must be taken to 
avoid misuse and misinterpretation. The value of indicators, 
Power argued/ is that they provide signals of outstanding 
questions to be resolved or potential problems that can then . be 
subjected to more intensive scrutiny. Specific indicator areas 
that might be further developed were presented under the 
following headings participation, educational attainments, 
measures of equity, attitudes towards education, quality of 
school life, socialization measures, and preparation for life. 
Not all the suggested indicator areas would necessarily be 
suitable for regular monitoring and might instead be addressed 
through periodic surveys. Either way. Power emphasized the 
practical need to identify statistics and indicators already 
av2&J:iable as a precondition of agreement on priorities for 
further developmental work. 

17. That levels of educational participation are bench-marks 
that often carry the possibility of substantial political impact 
was reported by a number of countries though against the 
background of the importance of defining exactly what 
"participation" is, and in which programs. The participation of 
different social groups of the student body is one priority area, 
it was agreed, but indicators . might equally show what is 
specially available for minority groups in addition to their 
participation in mainstream provision. Participation in such 
extra-curricular activities as, for example, museum-going might 
usefully be considered. There might be an incompatibility of 
different goals for improving participation^ levels of under- 
represented groups, such as lowering drop-oiit rates in schools 
and increasing enrollment levels in third-level education, that 
could render problematic the interpretation of measures. Given 
that a substantial body of statistics and information on 
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educational participation exists in many countries, the task for 
further work might "most usefully be to choose among available 
measures with comparability considerations uppermost, one 
^ delegate suggested. 

18. There were a number of specific recommendations made for 
possible directions of further work. One that emerged, 
particularly from those representing francophone counties, was 
that progress of a evatistically "fictitious" cohort through the 
education system can be assessed and compared using available 
statistics that would show flows' and attrition, taking account of 
i sui^ factors as migration and death rates. Even so, cross- 

na€ipnal comparisons ^ould probably not be able to reflect 
adeqpiately differences in curriculum contents as opposed to 
similarities and differences of career paths. Another delegate 
warned of linking indicators of participation too closely with 
access to, and accpiisition of, qualifications, that are multi- 
farious and often highly country-specific. Completion rates of 
education cycles could well provide a more meaningful alter- 
native, he suggested. What happens to students after they leave 
schooling — in terms of entry to employment, to post-secondary 
education and training, and to life-long learning opportunities 
iaore generally — was regarded by several as important areas that 
risk being neglected if there is an overly exclusive focus on 
schools and schooling. 



IV.- RESOURCES FOR SCHOOLING' 

What are the difficulties in measuring and comparing 
fiscal resources across countries? Can a "fiscal effort" 
index be devised that will account for international 
differences in economies, currencies and population sizes? 
What other resources, such as time devoted to learning, 
need to be explored? What does research tell us about how 
other resources affect the quality of schooling and the 
learning outcomes for students? Are comparedDle data 
across countries possible to obtain? 

19. This session was divided into two parts with introductory 
papers by: 

— Francois Orivel, IREDU, University of Dijon, 
Resources — How Much/How Many? 

— Linda Darling-Hammond, The Rand Corporatd.on, 
California, Resources — How Good/How Well Used? 

The discussants of these presentations were, respectively, 
Frederick Plank (Austria) and Eric Bolton (U.K.) 

20, Francois Orivel focused his presentation on a number of 
the methodological issues that arise with the measurement of 
educational resources and that should be confronted in the effort 
to refine and develop indicators of them. The refinement of 



these indicators, he argued, is especially necessary duriiiv^ this 
periled of budgetary constraints when education ministries need to 
be able to defend themselves againr ' pressure for further 
cutbacks fthough one delegate queried v^iether good measures of, 
for example, educational spending as a share of GKP would achieve 
this end) • The very concept of expenditure is not always clear - 
• apart from direct spending on such it ims as buildings and 
equipment, there. are Indirect items such as food and transport, 
other efforts (e.g. time spent by parents on homework) that 
frequently escape notice, as well as .le earnings foregone factor* 
when individuals extend their education « The * puLlic/private 
distinction,^ both in terms of institutions and expenditures, Xs 
also conceptually fraught, and decisions must be made about what 
should^ be included under each. This applies particularly to 
international comparisons since countries often use different 
concepts and accounting mechanisms for both public and private 
outlays. Differing cost burdens must also enter the picture of. 
the total reso:irce effort since these determine, to f^ome degree, 
expenditure levels. Concerning the indicator of teacher' 
salaries, Orivel showed how many factors enter the comprehensible 
assessment of remuneration, including the comparison of teachers 
with other occupations, which should, he asserted, assess 
absolute sums over a career with cost of living discoiinted. 
While analysis of teacher remuneration is certainly possible, 
Orivel noted, it is both a very sensitive are^ politically and 
not necessarily the best indicator of the attractiveriess of 
teaching. 

21. The discussion raised numerous technic; \ points and 
particular country practices concerning mea arement and 
accounting were reported. Many delegates agre d both that 
international efforts in this area are worthwhile and that 
definitional/methodological questions need to be clearly 
addressed to avoid serious misinterpretation. Several expressed 
doubts about the validity of ad iifls surveys and "grass roots" 
data of, for example, family expenditure in order to gauge 
private spending. More than one speaker also placed priority on 
the need to link expenditure data to outcomes in order to derive 
measures of efficiency and effectiveness. The difficulty of so 
doing was not underestimated nor tha complexity of 
interpretation, faced with the apparently contradictory message 
from research that financial resources are both one of 
education's cirucial ingredients yet alongside a surprisingly weak 
correlation between differences in spending and in measured 
outcomes . 

22. A number of very specific topics were also addressed « 
Attention was given to trends in pupil-teacher ratios and their 
adequacy as measures of teaching resources, (compared with others 
such as average class size) , as it was to teacher supply and the 
foreseen shortages in some countries in certain school subjects. 
The level of tuition fees, the degree to which this acts as a 
barrier to student participation in further education, and the 
redistributive impact (or lack of it) of education, were raised 
as possible subjects for indicator development. • Spending on 
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recurrent education and training was viewed ' as an area supported 
by a( notbripusly weedc data base yet representing a vital part of 
overall learning opportunities. And another suggested that one 
inteiresting. indicator of "healthy" educational expenditure would 
be tlie proportion spent ay a system on innovation or on develop- 
ing hew 'channels of information. 

23. Linda Darling-Hammond discussed the "qualitative" aspects 
of resources under three maiii headings: teachers (Wio? What? 
How; uisted?), time (for instruction, in relation to teachers and 
studehts and for in*^ .raction between them), opportunities to 
learn (the application of the curriculxm to different groups of 
students). DarlingrHammbnd organized her presentation around a 
number of indicator areas that, could elicit further interesst, 
though Che drew attention to those that appeared more suitable to 
rese^trch inquiiry or to national, as opposed to international, 
development. Concerning teachers, she argued that certain areas, 
such as certification and teacher behaviors, are either extrenely 
complex or subject to widely differing practices. Teachers' 
background education; experience, and in-service education should 
all enter the description of overall teacher resources, and 
priority should be given to how schools use teachers for 
instruction, with "who gets the good teachers?-^ (the equity 
issue) ah important additional aspect.- The time fa'stor was seen 
as worthy of further work, though again with certain areas, such 
as hoicework and how time is actually spent in class, presenting 
formidable conplexities, as does the concept of "opportunities to 
learn". Darling-Hammond considered that class and school size 
could well be investigated as indicators of the "personalization" 
of education. 

24. High quality teaching was underlined in the discussion as 
perhaps the key resource availeOale to schools and a niamber of the 
findings about what is known about effective teachers was 
reported — mastery of subject matter, effective management of 
learning, high expectations for all pupils and for themselves, an 
affinity for young people. How readily these can be translated 
into guidelines and indicators is far from clear and a long-term 
enterprise, howe^:er, thohgh the target group for improvement, it 
was suggested, should be the large group of teachers who are at 
neither extreme of the excelleiit or very weak. The long-term 
nature of inquiry and the need for research into the issues 
discussed during this session were reiterated by several 
speakers, whether it was concerning the teacher's changing role 
in the light of alternative sources of information and the media, 
or in ascertaining relationships between inputs and outputs. One 
delegate emphasized the need for further work on school 
principals though Darling-Hammond considered the evidence on 
effective school leadership to be so various as to cast doubt on 
this specific topic as orte suitable for indicator development, 
especially at the international level. 
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. V. THE C0NTE5CT WITHIN WHICH SCHOOLING TAKES PLACE. 

What are the external factors that most affect the 
learning environment? Which of those fact can or 
cannot be altered by governing entities? What information 
would be most useful to help, decision-makers institute 
policies that could have the most positive impact on 
learning outcomes? 

25* The paper introducing this session was given by James 
' Coleman of the University of .Chicago, with Alan Ruby, Department 
of Education, New South Wales, Australia, as discussant* 

26* James Coleman located consideration of the relative 

influence of schools and family background in the recent history 
of discussion of this topic, in which his own report in the 1960s 
had had a decisive influence. Evidence shows that schools are 
more effective for those from "good" backgrounds. The 
interactions between these various influences are as worthy of 
attention, he asserted, as their "independent effect" — when 
there are few available educational resources, their level and 
use have considerably greater relative impact than .when an 
education system is highly developed. In the latter ca&o, it is 
the favorable family backgromid that is the ^limiting factor of 
short supply? determining the interaction between the two and 
hence the critical ingredient of the two for outcomes* This he 
termed "social capital" (in contradistinction to the economic 
concept of "himan capital") and this was defined as the norms, 
values, and relational fstanictures that reside in the family and 
wider community* His research had indicated that a supportive 
commtinity cem compensate for the lack of "social capital" within 
the narrower aanbit of the family. A major woin^f he had was the 
widespread destruction of that "social capital", both in families 
and, even more, in communities. Conceiming indicators, Coleman 
argued for the usual measures of the "human capital" of families 
to be complemented by those of "social capital". 

27. Various speakers underlined the importance of this broad 
area as being indispensable if the "net" value of schooling is to 
be properly ascertained. The need to incorporate measures of 
poverty, and the question of whether statistics should be 
disaggregated by the ethnic/cultural origin of students, were 
recognized as important though highly sensitive questions. The 
problems^ faced by certain immigrant and minority youngsters, and 
the impact these have on their schooling, were, however, 
undoubted. How far there, should be direct public interventions 
in the "private" lives of faanilies and communities, or whether 
the role of policy is that of facilitating choices was raised as 
a question and one speaker identified the exercise of choice as 
the essential teature of "capital" of whatever sort. On the 
other hand, another speaker pointed to the possible contradiction 
between enhancing individual choice and stemming the destruction 
ot "social capital", since the individualism of the former had 
been identified by Coleman as a very cause of that destruction. 
Coleman attested that competition can have very beneficial or 
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^ harmful effects and that its exercise within a cooperativ.e 
setting is much more effective than individual competition • 



VI* CONCLUSIONS 



28* The final session provided delegates with the opportxinity 
for expressing their general conclusions and observations derived 
from the meeting, to consider how the ongoing work in the OECD, 
both on educational statistics and in other fields, could be 
exploited and further developed, t^o react to a working list of 
indicator items circulated during the meeting, and to suggest how 
the results of the conference could be followed up in the future. 

29. The general comments of delegates reflected a range of 
opinions as to the value and uses of indicators , and matters of 
comparability «md priority, though there was a broad measure of 
agreement on the value of further work on indicators . These 
comments included the following: 



Several Member countries expressed strong interest in 
student cognitive achievement indicators, citing the 
needs, of education ministries emd other policy-makers 
for this type of performemce measiire and the eagerness 
of the general public to obtain such infoinnation. 
Others did .not find student cognitive achievement a 
matter of priority or primary consequence. However, 
there appeared to be broad agreement on the usefulness 
of indicators of student participation, student 
completion of cycles and of equal opportunity for 
educational services as key measures of the performance 
of education. 

— The "health" of education ~ that indicators 
measure — is perceived differently from country to 
coimtry and inextricably linked to the goals in each. 
Even so, much useful exchsoige of data and information 
can take place even in the absence of international 
concurrence on such values or goals. 

— The needs of Member countries differ, and the uses of 
indicators exchanged among them would differ 
accordingly. The general feeling of the conference was 
that a number of countries expect to find sufficient 
benefits from the exchange of indicator data, and of 
the methods and measures used, that efforts should be 
made to develop selected international education 
indicators. Indicators can be useful for comparative 
purposes by helping to identify features or 
circumstances in a country that might be examined more 
closely for mpdiiy.cation or jjpprovement . The 
significant comparison in this case may be the relative 



rates of change of results over time from one country 
to another r rather than direct comparisons of absolute 
levels or rates. But for some countries ^ selecting 
indicators and collecting data may also be of 
assistance for national planning or evaluation and 
these may well value most the exchange of information 
on concepts, methodologies, and methods for this 
purpose. 

~ Cbncemihg the responses of the delegates to the 

working list />f indicator items circulated during the 
meeting, the 'items rated as higher priority clustered 
into two areas ?~ first, outcomes indicators on student 
learning performances, participation and completion; 
second, indicators of school processes and curricular 
offerings. The items given low interest clustered into 
• the area of socialization, such as indicators of school 
delinquency or substance abuse at school. 

— Several additional indicators were suggested by 

delegates euid numerous notes, comments, and clarifying 
statements were added to the original list of items. 
These notes should prove most useful in follow-up work 
by Member countries. 

~ certjain of the fxmdamental observations made throughout 
the conference — the dependence of indicators on 
particular values and models, the need to distinguish 
statistics and research, whether to develop a broad or 
narrow rauige of indicators, ' the need to be clear about 
the level of education being addressed and uses to 
which indicators are to be put — were also reiterated 
in the concluding session. 

30. Monique Solliliage, of the OECD Secretariat, described the 
statistical activities and data bank at the OECD, the degree to 
which these already generate some of the indicators discussed at 
the meeting or to which the data bank could be so used if 
desired, as well as some general observations about international 
comparisons of statistics and indicators. George Papadopoulos 
complemented this presentation by reference to other activities 
of the Education Committee and CERj that have produced, or will 
in the future, indicators and comparaQ3le findings on specific 
topics — e.g. teachers, the educational attainment of the labor 
force, the educational experiences of migrant and minority 
children — that might also contribute to further work on 
indicators. 



31. Educational statistics already calculated for publication 
by -the OECD, were presented by Monique Solliliage in describing 
the forthcoming volume. Education in OECD Countries in 1984-85, A 
Compendium of statistical Inforaation. In that publication, 
differences stnd' similarities among countries are to be 
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highlighted. (Information to be included was listed in full in a 
paper prepared by the Secretariat for the conference). Very 
broadly,, the following were described; 

-f Pupil enrollment in relation to population by level, 
full- and part-time, sex, general and 
technical/vocational, and changes 1983/84 to 1984/85; 

. ~ GDP devoted to public expenditure on education and 
expenditure by level of education and per pupil; 

— Teacher full-time equivalents by sex and level; 

— Student enrollment at the third-level showing new 
entrants, age, and sex; 

— Enrollment rates by single year of age; 

— Qualifications (or completions) at the second- level, 
general and technical/vocational and those eligible for 
access to third-level by sex and completions at the 
third-level ; 

— Expenditures by public and private sources by current 
. and capital and purpose (administration, teaching 

staff, books, etc.) and showing change, 1983/84 to 
1984/85. 

Other areas that are covered, at least partially, by the annual 
gathering of statistics through the joint OECD/UNESCO/SOEC 
questionnaires and other sources and that might be further 
developed include: 

— Completion rates for compulsory schooling; 

— Post school experience; 

— Fiscal effort (expenditures in relation to GDP) ; 

— Student financial aid for post-compulsory schooling; 

— Pupil-teacher ratios; 

— Population projections; 

— Unemployment rates; 

~ The "tracking" of students into r»ifferent branches; 

— Age of starting and end of compulsory schooling. 

32. Delegates welcomed improvements already made in OECD 

statistical data collection and reporting in the field of 
education and suggested that this improvement be fully continued 
in the future. They were reminded that a precondition of good 
data and up-to-date reports is the rapid and full completion of 
the emnual questionnaires. Special attention was urged for more 
complete descriptions as to how each country ^s statistics compare 
with staindard OECD definitions in order that indicators and 
statistical tables be well iinderstood by all who use them. Hence, 
conference participants supported suggestions for additional and 
continuing attention to educational statistics through a regular 
nechanisfe to update and revise existing items as well as to add 
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new measures. OECD should also give assistance to countries on 
definitions^ data "crosswalks" and related technical matters^ 
with emphasis on making the data comparable. The Secretariat 
described plans for a forthcoming meeting of national statistical 
experts to review statistical items^ comparisons^ reports^ and 
country contributions. The details of this meeting have not yet 
been finalized but it is envisaged for the first half of 1988. 
it will be the first of its kind in some years. 

33 i Mot all the areas that were under consideration at the 
conference correspond to the regular data collection undertaken 
by the OECD, nor are they necessarily eunened>le to this form of 
compilation* One representative described current OECD activity 
in this field as a data collection that is stronger on 
"quantitative" than "qualitative" reporting. Subjects not 
covered by the OECD questionnaires but that might be appropriate 
for future indicator development as discussed at the conference 
include: 

Student achievement; 

— Community expectations; 

— Resources for disadvantaged students; 

~ Private expenditures on vocational education; 

~ Supply and demand for student "places" in vocational 

program; 
~ Teacher characteristics;. 

'~ Resources for school activities in addition to teacher 
salaries; 

Teacher attendance and turnover rates; 
~ "Opportunity to learn" in selected curricular areas; 
~ Diversity in the school population; 
— > Substance abuse in schools; 

The use of time at school; 
~ Teacher "in-service" training. 

34. A number of the concluding remarks addressed one or more 
of the above areas in terms of potential follow-up work on 
education indicators in the international conliext. These 
suggestions included: 

i) The need to develop comparable data on student 
achievement, the quality of teaching, teacher 
character istifcs, and school level context and 
processes ; 

ii) There could be a very useful international study on 

what each interested Member country regards as adequate 
schooling and how that adequacy is judged or assessed; 

iii) Studies under the auspices of lEA (International 

AssociaticA for the Evaluation of Educational Achieve- 
ment) may be the most appropriate way to advance 
reporting on many aspects of student achievement as 
well a3 school, teacher, and curricular issues related 
to achievement. Closer links between the work of OECD 
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and the lEA would, be desireible as well as possible bl- 
and BUlti-lateral relations between national assessment 
agencies and/or inspectorates ' and the OECD. 

iv) OECD^s education research arm, CERI, should be well 
placed to examine issues of curriculum content, 
country-by-country, again as a means for broadening and 
deepening the store of qualitative data on schooling. 

35* Concerning procedural steps to be taken to build on the 
interest and progress of the Washington conference, George 
Papadopoulos for the Secretariat reported that the summary report 
of the conference (this document) would be distributed to the 
Education Committee at the end of November 1987, for discussion 
and their agreement on follow-up work within the Committee's 
activities* In addition, the Education Committee would be 
considering, at the same meeting, proposals for future work on 
the Quality of Schooling that overlap substantially with many of 
the subjects and issues discussed at the conference* He also 
reminded delegates of the proposed 1988 meeting envisaged on 
educational statistics, of the other activities that could well 
contribute particular expertise and indicators to the overall 
effort, and the potential role of CERI for further research and 
development on several of these topics. 

36. A number of procedural steps for the follow-up work was 
suggeisted by dele^gates. These included: 

i) The OECD should devise mechanisms and procedures, and 
should devote the requisite resources, to support the 
development of education indicators in addition to 
its regular statistical data collection and reporting. 
The latter should be prompt and up-to-date and made 
subject to ongoing review on a regular basis; 

ii) A task force of representatives of interested Member 
countries could be established to draw up and refine 
proposals for indicators and follow-up actions, with 
the aim of re-convening a fojnam along the lines of the 
joint OECD-US Washington conference. 

iii) The OECD, through a task force such as the one referred 
to above, could develop a tentative list of indicators, 
sufficiently broad to encompass varying interests, and 
circulate it for comment by Member countries; 

iv) Special bi-national or multi-national groups could be 
created to carry out developmental work on specific 
indicators, or clusters of indicators, according to 
their special interests. There should be the 
possibility for different countries to take the lead 
(through funds, staff, meeting i;ponsorship) for 
specific indicators or clusters, corresponding to those 
interests . 
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f ' y) A "learning network" of correspondents from Member 

. countries could be estaO^lished which would maintain 

( communications on issues of :i:autual interest in 

planning, decision-making, and evaluation as well as 

matters of comparability; 

: ^ vi) The OECD Secretariat and Member countries should fully 

: * exploit the potential of existing international 

r computerized links for data exchange and remain abreast 

: . ' of future developments. 

vii) Supplementary resources should be sought for future 
( work but the aibsence of such resources should not 

prevent progress in this field. The matter is of 
sufficient importance that some reallocation of 
C existing resources would, if necessary, be warranted* 
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EPILOGUE 



Peter IL Greer 

Deputy Under Sectretary for Intergovernmental and Interagency Affairs 



During the concluding discussions of the Washington 
Conference, It bec2usie clear that there Is widespread interest in 
education Indicators. Also, there Is general agreement that the 
task of developing these Indicators will be complex and will have 
to take Into consideration the different organizational patterns 
and practices In education of the Interested countries. 

The recommendations of the Washington Conference were 
presented to the OECD Education Committee and the Governing Board 
of Center for Educational Research and Innovation (CERI) In 
November and December 1987* Responsibility for further 
development of the conceptual Issues was assigned to CERI while 
further development of educational statistics and policy issues 
was to-be continued by the Education Committee. In addition, the 
French Ministry of Education offered to host a second meeting, in 
the spring of 1988, to continue the process of developing 
comparable international education indicators. Also, the 
U.S. Department of Education agreed to provide further support to 
the OECD Secretariat to mcOce availed^le additional staff resources 
to assist with the project. 



Peter R. Greer 
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AGENDA 

International Conference' on Cross-National Education Indicators 



ARRIVAL (Tuesday November 3, 1987) 

Registration, 



3:30 - 5:30 p.m. 
7:00 p.m* 



Ramada Renaissance 
Hotel 



Reception 



WELCOME TO THE CONFERENCE 

Emerson J* Elliott, Director 
Center for Education Statistics 
U.S. Department of Education 



WEDNESDAY, Nov. 4 



Pan American Health Organization Building 
525 Twenty-Third Street, N.W. 



9:00 - 9:20 a.m- 



INTRODUCTIONS AND OPENING REMARKS 

Peter Greer, Deputy Under Secretary for 
Intergovernmental and Interagency 
Affairs 

U.S. Department of Education 

George Papadopoulos, OSCD Secretariat 
Deputy Director for Education, OECD 

Emerson J. Elliott, Director 
Center for Education Statistics 
U.S. Department of Education 
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SESSION I. 



INDICATORS: CONCEPTS AND MEASUREMENT 
ISSUES - • 



What are education indicators? How can they 
be used? What kinds of issues do they raise 
for various groups involved in education? 
What are the optiiial components of a system 
of indicators? 

Chairman: wilmer Cody 

Council of Chief State 
School Officers 



9:20 ^ 10:15 a.m. Presentation: 

Jeannie Oakes 

The Rand Corporation 

"Education Indicators — Concept, 
Types, Uses'* 

(A broad conceptual introduction' to 
the notion of education indicators 
raising issues about their use, and 
potential misuse at various levels 
with, particular attention to 
international concerns) 

Discussion 

How well do these concepts 
apply to other OECD countries? 



10:15 - 10:30 a.m. Coffee break 

10:30 - 12:00 noon Presentation: 

Neville Postlethwaite 
University of Hamburg 

"Methodological Issues in Indicator 
Development on the International Level" 

(Examples of major technical issues 
facing an attempt to derive 
comparable data across the OECD 
countries) 



Discussion 
12:00 • 1:30 p.m* Liinch Break 
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ERjc ; 



Presentation: 
Philip Halsey, 

Deputy Secretary^ Department of 
Education and Science, United Kingdom 

"Education Indicators: Concepts, 
Measures and Educational Policy 
in England and Wales 

Discussion 



Coffee break 



INDICATORS OF THE OUTCOMES OF EDUCATION - 
What are the best sources of data on 
student performance in school? What are 
the major issues regarding comparability 
of scores across countries? What other 
outcomes of schooling need to be explored? 



Chairman: Herbert Walberg 

University of Illinois 



, Presentation: 



Alan Purves, Chairman, International 
Association for the Evaluation of 
Educational Achievement 

"Student Performance - lEA Studies" 

(What are the short and long-term 
issues concerning the potential for 
use of lEA studies as a source of an 
outcome measure and whether the 
countries can/ in the future, work 
toward certain shared test items in 
their own testing programs as an 
alternative to or supplement to lEA 
studies?) 

Discussion 




4:30 - 6:30 p.m* BREAK 

6:30 - 7:00 p.m* RECEPTION AND CASH BAR Ramada 

Renaissance 



7:00 p.m. DINNER 



Hotel 



8:00 p.iQ* SPEAKER: Chester E* Finn, Jr. 

Assistant Secretary for 
Educational Research and 
Improvement and Counselor to 
the Secretary 

Address: "U.S. Education Reform and 
International Data" 

THURSDAY, NOV. 5 

9:00— 10:30 a.m. Presentation: 

Colin Power 

Flinders University, Australia 

••Outcomes - Other Possible Measures" 

(What are the important outcomes of 
schooling other than cognitive 
achievement? The discussion may 
include traditional indicators 
such as completion of school, 
enrolling in higher education or 
additional training and non- 
tradiitional measures such as 
attitudes) 

Discussion 

10:30 - 10:45 a.m. Coffee break 
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• . SESSION III RESOURCES FOR SCHOOLING- - What are the 

difficulties in measuring and comparing 
fiscal resources across countries? Can a 
"fiscal effort" index be devised that will 
account for international differences in 
economies, currencies, and population sizes? 
What other resources, such as time devoted 
to. learning, need to be explored? What does 
research toll us about how other resources 
affect the quality of schooling and the 
learning outcomes for students? Is • 
comparable data across countries possible to 
obtain? 



Chairman: Stephen Heyneman, Chief 

Education and Training Division 
The World Bank 

10:45 ~ 12:00 noon Presentation 



Fremcois Orivel 

IREDU, University of Dijon 

"Resources How Much/How Many?" 

(What cros?-national comparisons- of 
educational finance can be conducted? 
What provisions are made for 
measurement of human resources such as 
teacher supply, pupil/teacher ratio, 
teacher/administrator ratio?) 

Discussion 

12:00 - 1:30 p.m. Lunch Break 

1:30 - 3:00 p.m. Presentation: 



Linda Darling-^Hammond 
The Rand Corporation 



"Resources — How Good/How Well Used?" 

(What are some measures of teacher 
quality, time use, and opportunity to 
learn that might be used cross- 
nationally?) 



Discussion 



3:00 - 3:15 p.m. Break 

I 

£^ 37 40 



SESSION IV 



THE CONTEXT WITHIN WHICH SCHOOLING TAKES 
PLACE - What are the external factors that 
most affect the learning environment? 
Which of those factors can or cannot be 
altered by governing entities? What 
information would be most useful to help 
decision-makers institute policies that 
could have the most positive impact on 
learning outcomes? 

Chairman: Raimund Ritter 
Federal Republic of Germany 



3:15 - 4:30 p.m. 



Presentation: 

Jaunes Coleman 
University of Chicago 

•^Student Population Characteristics 
and Other Environmental Factors" 

Discussion 



FRIDAY, NOV. 6 
SESSION V 



9:00 - 10:15 a.m. 

10:15 - 10:30 a.m. 
10:30 - 12:00 noon 



FINAL SUMMING UP OF NEXT STEPS TOWARD 
INTEFIATIONAL COOPERATION - What have we 
heard in these deliberations about the 
feasibility of cooperation in this 
endeavor? What technical steps need to be 
taken to collect, exchange and analyze 
comparable data across countries? How 
will that role be implemented? 

Chairrcan: Smerson J. Elliott, Director 

Center for Education Statistics 
U.S. Department of Education 

Discussion 
Coffee break 
Plenary Session 



12:00 noon 



End of Conference 
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PAPERS 

International Conforencc on Cross-National Education Indicators 



Darling-Haxnmond, Linda, "Resources for Schooling: Teachers, 
Time, and Opportunities to Learn" 

Eide, Kjell, "The Need for Statistical Indicators in Education" 

Halsey, P.H*, "Educational Indicators: Concepts, Measures and 
Educational Policy in England and Hales" 

Orivel, Francois, "Some Thoughts and Suggestions for Drawing Up 
Internationally Stemdardized Indicators for Resources Allocated 
for Education" 

Postlethwaite, T. Neville^ "Methodological Issues in Indicator 
Development at the International Level" 

Power, Colin,' "Indicators of Outcomes of Education Other than 
Cognitive Achievement" 

Purves, Alan C, "Student Performance as an Educational 
Indicator" 
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